----------------------------------------------------------------------------
                    Understanding Comm Port Overruns
                        Last updated on 3-28-95
                  Send corrections and suggestions to:
                 dave.houston@ccc-bbs.com (Dave Houston)  
----------------------------------------------------------------------------
The Microsoft Knowledge Base article "Windows Support of the 16550 UART", 
at http://www.microsoft.com:80/pages/kb/PEROPSYS/windows/Q119579.htm
gives a clear and concise but somewhat terse explanation of the fundamental
cause of comm port overruns. This article is my attempt at clarifying the
subject for those for whom the Microsoft article may be insufficient.

----------------------------MODEM BASICS---------------------------------

+-------+     +---------+                      +---------+     +-------+
:       :     :         :                      :         :     :       : 
: CPU A :-DTE-: MODEM A :----TELEPHONE LINE----: MODEM B :-DTE-: CPU B :
:       :     :         :         DCE          :         :     :       : 
+-------+     +---------+                      +---------+     +-------+

---------------------------BASIC TERMINOLOGY------------------------------

MODEM (MOdulator/DEModulator) accepts the digital data supplied by the 
transmitting CPU and converts it to a modulated analog wave form that can
be transmitted over the phone line and accepts a modulated analog wave from
the telephone line, converts it to a digital form and passes it on to the 
receiving CPU. Modern, high speed modems incorporate error correction and
data compression. 

DTE (Data Terminal Equipment) - the local or remote computer (or terminal).
This term is also used to refer to the rate (in bits per second) that data
is passed between the CPUs and Modems via the computer's data bus. Most 
modems (by default) report this rate in CONNECT XXXXX upon making a 
connection to another modem. 

DCE (Data Communications Equipment) - the local or remote modem. This term 
is also used to refer to the efective Carrier Speed. Most modems can be 
configured to report this rate in CARRIER XXXXX upon making a connection to 
another modem.

DATA COMPRESSION is a technique whereby the transmitting modem examines the
data stream for redundancy and replaces redundant strings (repeat groups of 
characters) with special codes that the receiving modem interprets and uses
to restore the the data to it's original, uncompressed form. The compression
(and decompression) take place in the modems and allows for more efficient
transmission as more data can be transmitted in a given time period. A side
effect is that the effective rate between the modems (DCE) is more efficient
than the rate between the modems and CPUs (DTE). For this reason, the DTE 
rate should be set to 3-4 times the DCE rate. 

UART (Universal Asynchronous Receiver Transmitter) accepts parallel data 
from the computer's data bus and converts it to serial form for 
transmission via the serial port and accepts serial data from the serial 
port and converts it to parallel form for passing to the CPU via the 
computers parallel data bus. Older systems have 8250 or 16450 UARTs; newer,
high speed modems require 16550 UARTs. The 8250 & 16450 incorporate a one 
byte buffer; the 16550 has 16 byte FIFO (First In First Out) buffers and 
supports adjustable trigger levels (the point at which an IRQ is issued) for
both. For internal modems, the UART is on the modem board. Most high speed 
internal modems have (or emulate) a 16550 UART. External modems interface 
with a UART on the computer's serial I/O board. The 8250 and 16450 are 
essentially the same (Most diagnostics programs cannot tell them apart.) 
except that the 16450 is faster. In addition to the FIFO buffers and 
adjustable trigger levels, the 16550 is about 25% faster than the 16450. 
(See National Semiconductor's Application Note AN-491, ``The 16550A:
UART Design and Application Considerations'')

IRQ (Interrupt ReQuest) - a hardware signal that tells the CPU that it needs 
to service the device that issued the IRQ. In this case, it is how the modem
signals the CPU that there is data waiting in the UART buffer for the CPU to
retrieve. For each IRQ, the system BIOS contains canned code that the CPU
executes when it responds to the particular IRQ. Device drivers may modify 
or relocate (leaving a pointer or forwarding address) this code. Memory 
managers may relocate BIOS code as well, sometimes with untoward effects.

RTS/CTS (Request To Send/Clear To Send) hardware flow control. RTS and CTS 
are two control signal lines between the DTE and DCE that are used to 
control the flow of data between the DTE and DCE. THIS SHOULD ALWAYS BE USED
ON BOTH ENDS OF THE LINK.

XON/XOFF software flow control. A method of data flow control that uses 
special control characters embedded in the data stream. This will not work 
with binary data as the control characters cannot be distinguished from 
data. THIS SHOULD NEVER BE USED.

---------------------WHAT CAUSES DATA OVERRUNS?---------------------------

A data overrun occurs when the CPU fails to retrieve the data that is in the
UART buffer before it is overwritten by newly received data.

With a single tasking system (such as DOS without any multi-tasking) the CPU
usually has little else to do and has no problem keeping up with the data 
flow even with the single byte buffer of the 8250 & 16450 UARTs. As each 
byte is received the modem issues an IRQ and the CPU immediately responds 
and retrieves the waiting byte. In this environment, overruns are seldom 
experienced even when using high data rates and 8250 or 16450 UARTS.

But, in a multi-tasking environment such as Windows, the CPU is dividing its
attention to deal with all of the tasks that may be operating. It may be 
tied up elsewhere when the modem issues an IRQ and be unable to retrieve the
data in time to prevent an overrun. The larger FIFO buffer of the 16550 UART
with the adjustable trigger level alleviates but may not eliminate data 
overruns.

Because the 16550's FIFO buffer can hold more data (up to 16 bytes) the CPU 
does not have to service the port as often. This allows the CPU to devote 
more time to any other tasks that may be running. At the recommended RX
trigger of 8, an IRQ is issued when the buffer has accumulated 8 bytes and 
the FIFO can accept another 8 bytes before experiencing an overrun. In 
comparison to the older single byte buffer UARTs, the CPU now only has to 
service the port 1/8 as often and has 8 times as much time to do so. Alas, 
this is often not enough.

(The above is greatly simplified because it ignores the overhead involved in
servicing the port. The setup prior to retrieving the data and the clean up 
after takes far more time than the actual time that it takes the CPU to get 
the data and pass it on to a memory buffer where it can be accessed by the 
comm application program. National Semiconductor says that tests have shown 
that NS16550A will increase the performance by a factor of five at the 
system level over the NS16450.)

NOTE: Some have reported that they eliminated overruns by setting the RX
      trigger to 1. This issues a IRQ after each byte is received but allows
      15 more bytes to come in before there is an overrun. This seems a bit 
      drastic as there is some overhead associated with each fetch operation
      but it may work for some. I would suggest trying settings of 8, 4, and
      then 1 as a last resort. Given that on my relatively slow system, I
      did not get overruns with RX at 8, I am inclined to seek the cause 
      elsewhare.

Several things may prevent the CPU from retrieving the data in time. 

Some are:

1. TSR programs that disable IRQs. Microsoft's SPEAKER.DRV that allows .WAV
   files to be played through the PC speaker without a sound card is one 
   such TSR. There are others. (There is another SPEAKER.DRV available on 
   CompuServe, in the WinCIM support forum, that does not disable IRQs.) 
   Try removing all TSR's. If overruns cease, add them back one at a time 
   until you find the culprit.
   
2. Write caching in Smartdrv and other disk cache programs. The /X switch 
   will turn off write caching for Smartdrv. If you use WFWG 3.11 and 32-bit
   file access it has its own cache and Smartdrv is doing nothing but using 
   up memory. You should either remove it or set the Windows cache size to 
   zero.

3. Some memory managers move BIOS code out of the first megabyte of DOS
   memory. Some seem to do this better than others. I use Netroom and have 
   the System BIOS "cloaked" and do not experience any problems. Memmaker
   is a subset of Netroom code licensed by Microsoft from Helix. There are
   reports that QEMM prior to 7.5 has some problems. 
    
4. Video Accelerator cards that suspend IRQs for too long. Try using the VGA
   16 color mode; if it improves the situation then maybe a later driver for
   your video card will help.  

5. Poorly written programs that suspend IRQs for too long. Try removing all
   other running programs. If overruns cease, add them back one at a time 
   until you find the culprit.

6. The standard comm.drv supplied with all versions of Windows prior to 
   WFWG 3.11 does not adequately support the 16550 UART. WFXCOMM.DRV from 
   Delrina is freeware and widely available. It supports all of the features
   of the 16550 including adjustable trigger levels. CYBERCOM.DRV is also
   widely available but does not support adjustable trigger levels and has
   been reported to be incompatible with some fax and comm programs. For
   Windows 3.1, I can recommend WFXCOMM.DRV. Before switching to WFWG 3.11, 
   I used it and had no overruns.

   Others have reported good results with CYBERCOM.DRV but in all cases 
   where I have *direct* experience, it has either not stopped the overruns
   or has been incompatible with other fax or comm software.

7. Multiple reasons. Two or more of the above may be at fault. So don't 
   assume that, because you tried one of the suggestions above and still have
   overruns, that the feature you've just removed is absolved of blame. You 
   may have to implement two or more of the above suggestions.

---------------------MY WINSOCK SPECIFIC SETTINGS--------------------------

My system was a Gateway2000 386/25 with:
(I have since upgraded to a 486dx2/66 with a second WD Caviar HD and a VLB
 IDE-I/O card but the setup below is the one that I ran on the 386 with 
 no overruns.)
 
  Diamond Speedstar Pro (ISA) graphics accelerator (1Mb)
    (using 800x600x65,000 color driver)
  Boca v.FC 28.8 internal modem (16550 emulation)
     I get 28.8 connects with several BBSs (~3300 cps on ZIP downloads)
     and 28.8 connects with my ISP (~3.11 KBytes/sec ftp on ZIP)
     I see no overruns in either situation.
  WD Caviar EIDE HD (13 ms access)
  Generic ISA IDE-I/O  
  WFWG 3.11 32-bit Disk Access and 32-bit file access 
  Win32s v1.25 (v1.15 and 1.20 were OK also)

While surfing, I normally have running:

  ScreenPeace screensaver 
  SysGraph (graphs CPU load) 
  FreeMem (shows System Resources and Virtual Memory)
  Pegasus (minimized - checks for new mail on 10 minute intervals)
  Netscape 1.0 or Free Agent and/or WS_FTP32

  I am currently running PPP with the settings detailed below in the 
  MINE/PPP column. Previously, I ran SLIP with the settings in the
  MINE/SLIP column. I also ran PPP initially with the same settings 
  for MTU, RWIN & MSS as I was using for SLIP.

             ____MINE____        ____*PETER TATTAM____
             PPP     SLIP        SLIP          NETWORK 
     MTU:    576      552         252             1500
TCP RWIN:   8576     2048         848             4096
TCP  MSS:    536      512         212             1460
    BAUD: 115200** 115200**               
    SLIP:     no      yes
     PPP:    yes       no
   CSLIP:    yes      yes

 * Peter Tattam's recommended settings (see INSTALL.DOC v2.0b)

** hardware dependent - set it at the highest level that the modem
   can support unless it causes underruns on transmits. It should normally
   be 3-4 times DCE at minimum.

NOTE: There is some confusion re MTU as install.doc says it should be 
      TCP MSS + 40 but then recommends 256 with TCP MSS equal to 212.
      I have tried various settings for MTU, RWIN & MSS and except for
      getting PPP frame errors at MTU = 1500, I saw no obvious differences.
      I have found *no* settings that produce overruns so I am forced to
      conclude that the problems others are experiencing are probably not
      related to these settings as long as they are within recommended
      ranges.

-----SYSTEM.INI settings -----

WFXCOMM (Win 3.1)     COMM.DRV (WFWG 3.11)     CYBERCOM.DRV   

  [386enh]              [386enh]                 [386enh]
  COMxFIFO=1            COMxFIFO=1               COMxFIFO=1
  COMxRXSize=8          Rx default is 8          Rx fixed at 8
  COMxTXSize=16                                  Tx fixed at 8

NOTE: With WFXCOMM.DRV and WFWG's COMM.DRV, the RX trigger level can be set 
      to 1, 4, 8, or 14. While 8 is recommended (the default for WFWG)
      and works well for most cases, lower settings provide more leeway to
      prevent overruns at a cost of more frequent fetches and added 
      overhead.

I have also seen recommendations to set COMxBuffer=SomeMagicNumber but I am 
using the default (128) with no ill effects.

Finally, I can only say that the view expressed in the Overruns FAQ at:

   http://www.trumpet.com.au/wsk/faq/overruns.htm

that overruns are just a fact of life under Windows and that the only 
solution is to live with them is incorrect.

==========================================================================
References:

RFC  879 - The TCP Maximum Segment Size and Related Topics
RFC 1055 - Nonstandard for transmission of IP datagrams over serial links: 
           SLIP
RFC 1144 - Compressing TCP/IP Headers for Low Speed Serial Links
RFC 1331 - The Point-to-Point Protocol (PPP) for the Transmission of 
           Multi-protocol Datagrams over Point-to-Point Links
RFC 1332 - The PPP Internet Protocol Control Protocol
RFC 1661 - The Point-to-Point Protocol (PPP)

Available via ftp from ds.internic.net/rfc/rfc####.txt or rfc####.ps

    or

View at http://www.MorningStar.com/rfc.html

Windows Support of the 16550 UART
http://www.microsoft.com:80/pages/kb/PEROPSYS/windows/Q119579.htm
==========================================================================

